Face Generation

In this project, you'll define and train a DCGAN on a dataset of faces. Your goal is to get a generator network to generate new images of faces that look as realistic as possible!

The project will be broken down into a series of tasks from loading in data to defining and training adversarial networks. At the end of the notebook, you'll be able to visualize the results of your trained Generator to see how it performs; your generated samples should look like fairly realistic faces with small amounts of noise.

Get the Data

You'll be using the CelebFaces Attributes Dataset (CelebA) to train your adversarial networks.

This dataset is more complex than the number datasets (like MNIST or SVHN) you've been working with, and so, you should prepare to define deeper networks and train them for a longer time to get good results. It is suggested that you utilize a GPU for training.

Pre-processed Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. Some sample data is show below.

If you are working locally, you can download this data by clicking here

This is a zip file that you'll need to extract in the home directory of this notebook for further loading and processing. After extracting the data, you should be left with a directory of data processed_celeba_small/

In [1]:
data_dir = 'images/'

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np
import problem_unittests as tests
#import helper

%matplotlib inline

Visualize the CelebA Data

The CelebA dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations, you'll only need the images. Note that these are color images with 3 color channels (RGB)#RGB_Images) each.

Pre-process and Load the Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. This pre-processed dataset is a smaller subset of the very large CelebA data.

There are a few other steps that you'll need to transform this data and create a DataLoader.

Exercise: Complete the following get_dataloader function, such that it satisfies these requirements:

  • Your images should be square, Tensor images of size image_size x image_size in the x and y dimension.
  • Your function should return a DataLoader that shuffles and batches these Tensor images.

ImageFolder

To create a dataset given a directory of images, it's recommended that you use PyTorch's ImageFolder wrapper, with a root directory processed_celeba_small/ and data transformation passed in.

Data Citation

@inproceedings{liu2015faceattributes, title = {Deep Learning Face Attributes in the Wild}, author = {Liu, Ziwei and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou}, booktitle = {Proceedings of International Conference on Computer Vision (ICCV)}, month = {December}, year = {2015} }

In [2]:
# necessary imports
import torch
from torchvision import datasets
from torchvision import transforms
from torch.utils.data import DataLoader
In [3]:
def get_dataloader(batch_size, image_size, data_dir='images/'):
    """
    Batch the neural network data using DataLoader
    :param batch_size: The size of each batch; the number of images in a batch
    :param img_size: The square size of the image data (x, y)
    :param data_dir: Directory where image data is located
    :return: DataLoader with batched data
    """
    
    # TODO: Implement function and return a dataloader
    transformation = transforms.Compose([transforms.Resize(image_size),
                                     transforms.CenterCrop(image_size),
                                     transforms.ToTensor()])
    data = datasets.ImageFolder(data_dir,transform=transformation)
    loader = DataLoader(data,batch_size=batch_size,shuffle= True)
    return loader

Create a DataLoader

Exercise: Create a DataLoader celeba_train_loader with appropriate hyperparameters.

Call the above function and create a dataloader to view images.

  • You can decide on any reasonable batch_size parameter
  • Your image_size must be 32. Resizing the data to a smaller size will make for faster training, while still creating convincing images of faces!
In [4]:
# Define function hyperparameters
batch_size = 256
img_size = 32

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# Call your function and get a dataloader
celeba_train_loader = get_dataloader(batch_size, img_size)

Next, you can view some images! You should seen square images of somewhat-centered faces.

Note: You'll need to convert the Tensor images into a NumPy type and transpose the dimensions to correctly display an image, suggested imshow code is below, but it may not be perfect.

In [5]:
# helper display function
def imshow(img):
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# obtain one batch of training images
dataiter = iter(celeba_train_loader)
images, _ = dataiter.next() # _ for no labels

# plot the images in the batch, along with the corresponding labels
fig = plt.figure(figsize=(20, 4))
plot_size=20
for idx in np.arange(plot_size):
    ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])

Exercise: Pre-process your image data and scale it to a pixel range of -1 to 1

You need to do a bit of pre-processing; you know that the output of a tanh activated generator will contain pixel values in a range from -1 to 1, and so, we need to rescale our training images to a range of -1 to 1. (Right now, they are in a range from 0-1.)

In [6]:
# TODO: Complete the scale function
def scale(x, feature_range=(-1, 1)):
    ''' Scale takes in an image x and returns that image, scaled
       with a feature_range of pixel values from -1 to 1. 
       This function assumes that the input x is already scaled from 0-1.'''
    # assume x is scaled to (0, 1)
    # scale to feature_range and return scaled x
    min,max = feature_range
    x = x * (max-min) + min
    return x
In [7]:
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# check scaled range
# should be close to -1 to 1
img = images[0]
scaled_img = scale(img)

print('Min: ', scaled_img.min())
print('Max: ', scaled_img.max())
Min:  tensor(-0.9137)
Max:  tensor(0.7020)

Define the Model

A GAN is comprised of two adversarial networks, a discriminator and a generator.

Discriminator

Your first task will be to define the discriminator. This is a convolutional classifier like you've built before, only without any maxpooling layers. To deal with this complex data, it's suggested you use a deep network with normalization. You are also allowed to create any helper functions that may be useful.

Exercise: Complete the Discriminator class

  • The inputs to the discriminator are 32x32x3 tensor images
  • The output should be a single value that will indicate whether a given image is real or fake
In [8]:
import torch.nn as nn
import torch.nn.functional as F
In [9]:
class Discriminator(nn.Module):

    def __init__(self, conv_dim):
        """
        Initialize the Discriminator Module
        :param conv_dim: The depth of the first convolutional layer
        """
        super(Discriminator, self).__init__()

        # complete init function
        self.conv_dim = conv_dim
        #define layers
        #first conv layer reducing dimensionality by 2 and increasing conv_dim to conv_dim
        self.conv1 = nn.Conv2d(3,conv_dim,4,padding=1,stride=2)
    
        #second conv layer reducing dimensionality by 2 and increasing conv_dim to 2
        self.conv2 = nn.Conv2d(conv_dim , conv_dim*2,4,padding=1,stride=2)
        #batch norm after 2nd conv layer to increase model performance
        self.bn2 = nn.BatchNorm2d(conv_dim*2)
 
        #third conv layer reducing dimensionality by 2 and increasing conv_dim to 4
        self.conv3 = nn.Conv2d(conv_dim*2,conv_dim*4,4,padding=1,stride=2)
        #batch norm after 3rd conv layer to increase model performance
        self.bn3 = nn.BatchNorm2d(conv_dim*4)

        #fourth conv layer reducing dimensionality by 2 and increasing conv_dim to 8
        self.conv4 = nn.Conv2d(conv_dim*4,conv_dim*8,4,padding=1,stride=2)
        #batch norm after 4th conv layer to increase model performance
        self.bn4 = nn.BatchNorm2d(conv_dim*8)

        #fifth conv layer reducing dimensionality by 2 and increasing conv_dim to 16
        self.conv5 = nn.Conv2d(conv_dim*8,conv_dim*16,4,padding=1,stride=2)
        #batch norm after 5th conv layer to increase model performance
        self.bn5 = nn.BatchNorm2d(conv_dim*16)
        #final linear layer with 1 output, BCEloss will be calculated so no need to apply sigmoid.
        self.fcout = nn.Linear(1*1*conv_dim*16,1)

        
        
        

    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: Discriminator logits; the output of the neural network
        """
        # define feedforward behavior
        #pass input tensor thru network
        x = F.leaky_relu(self.conv1(x))
  
        x = self.conv2(x)
        x = F.leaky_relu(self.bn2(x))
    
        x = self.conv3(x)
        x = F.leaky_relu(self.bn3(x))
    
        x = self.conv4(x)
        x = F.leaky_relu(self.bn4(x))

        x = self.conv5(x)
        x = F.leaky_relu(self.bn5(x))
        #reshape vector to fit into linear layer
        x = x.view(-1,self.conv_dim*16)
        out = self.fcout(x)
        #return logit of  linear layer
        return out


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(Discriminator)
Tests Passed

Generator

The generator should upsample an input and generate a new image of the same size as our training data 32x32x3. This should be mostly transpose convolutional layers with normalization applied to the outputs.

Exercise: Complete the Generator class

  • The inputs to the generator are vectors of some length z_size
  • The output should be a image of shape 32x32x3
In [10]:
class Generator(nn.Module):
    
    def __init__(self, z_size, conv_dim):
        """
        Initialize the Generator Module
        :param z_size: The length of the input latent vector, z
        :param conv_dim: The depth of the inputs to the *last* transpose convolutional layer
        """
        super(Generator, self).__init__()

        # complete init function
        self.z_size = z_size
        self.conv_dim = conv_dim
        #define layers
        #dense layer reshapping fron z_size to conv_dim*16
        self.fcin = nn.Linear(z_size,conv_dim*16)
        #first conv tranpose layer increasing dimensionality by 2 and decraising conv_dim to 8
        self.deconv1 = nn.ConvTranspose2d(conv_dim*16,conv_dim*8,4,padding=1,stride=2)
        #batch norm after 1st deconv layer to increase model performance
        self.bn1 = nn.BatchNorm2d(conv_dim*8)

        #second convtranspose  layer increasing dimensionality by 2 and decreasing conv_dim to 4
        self.deconv2 = nn.ConvTranspose2d(conv_dim*8,conv_dim*4,4,padding=1,stride=2)
        #batch norm after 2nd deconv layer to increase model performance
        self.bn2 = nn.BatchNorm2d(conv_dim*4)

        #third conv transpose layer increasing dimensionality by 2 and decreasing conv_dim to 2
        self.deconv3 = nn.ConvTranspose2d(conv_dim*4,conv_dim*2,4,padding=1,stride=2)
        #batch norm after 3rd deconv layer to increase model performance
        self.bn3 = nn.BatchNorm2d(conv_dim*2)


        #fourth conv transpose layer increasing dimensionality by 2 and decreasing conv_dim to conv_dim
        self.deconv4 = nn.ConvTranspose2d(conv_dim*2 , conv_dim,4,padding=1,stride=2)
        #batch norm after 4th conv layer to increase model performance
        self.bn4 = nn.BatchNorm2d(conv_dim)

        #last conv transpose layer increasing dimensionality by 2 and increasing conv_dim to 3
        self.deconv5 = nn.ConvTranspose2d(conv_dim,3,4,padding=1,stride=2)


    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: A 32x32x3 Tensor image as output
        """
        # define feedforward behavior
        #pass input thru first linear layer
        x = F.relu(self.fcin(x))
        #reshape to to 4d tensor to fit into conv layers
        x = x.view(-1,self.conv_dim*16,1,1)
        #pass thru conv layers
        x = F.relu(self.deconv1(x))
        x = self.deconv2(x)
        x = F.relu(self.bn2(x))
        x = self.deconv3(x)
        x = F.relu(self.bn3(x))
        x = self.deconv4(x)
        x = F.relu(self.bn4(x))
        #pass final output thru tan activation function
        out = F.tanh(self.deconv5(x))
        

        return out

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(Generator)
Tests Passed

Initialize the weights of your networks

To help your models converge, you should initialize the weights of the convolutional and linear layers in your model. From reading the original DCGAN paper, they say:

All weights were initialized from a zero-centered Normal distribution with standard deviation 0.02.

So, your next task will be to define a weight initialization function that does just this!

You can refer back to the lesson on weight initialization or even consult existing model code, such as that from the networks.py file in CycleGAN Github repository to help you complete this function.

Exercise: Complete the weight initialization function

  • This should initialize only convolutional and linear layers
  • Initialize the weights to a normal distribution, centered around 0, with a standard deviation of 0.02.
  • The bias terms, if they exist, may be left alone or set to 0.
In [11]:
def weights_init_normal(m):
    """
    Applies initial weights to certain layers in a model .
    The weights are taken from a normal distribution 
    with mean = 0, std dev = 0.02.
    :param m: A module or layer in a network    
    """
    # classname will be something like:
    # `Conv`, `BatchNorm2d`, `Linear`, etc.
    classname = m.__class__.__name__
    
    # TODO: Apply initial weights to convolutional and linear layers
    #class is a conv layer or linear layer, adjust weights
    if 'Conv' in classname or 'Linear' in classname:
        m.weight.data.uniform_(0.0, 0.02)
        m.bias.data.fill_(0)
    

Build complete network

Define your models' hyperparameters and instantiate the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments.

In [12]:
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
def build_network(d_conv_dim, g_conv_dim, z_size):
    # define discriminator and generator
    D = Discriminator(d_conv_dim)
    G = Generator(z_size=z_size, conv_dim=g_conv_dim)
    
    # initialize model weights
    D.apply(weights_init_normal)
    G.apply(weights_init_normal)

    print(D)
    print()
    print(G)
    
    return D, G

Exercise: Define model hyperparameters

In [13]:
# Define model hyperparams
d_conv_dim = 128
g_conv_dim = 128
z_size = 100

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
D, G = build_network(d_conv_dim, g_conv_dim, z_size)
Discriminator(
  (conv1): Conv2d(3, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (conv2): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (conv3): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (conv4): Conv2d(512, 1024, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn4): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (conv5): Conv2d(1024, 2048, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn5): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (fcout): Linear(in_features=2048, out_features=1, bias=True)
)

Generator(
  (fcin): Linear(in_features=100, out_features=2048, bias=True)
  (deconv1): ConvTranspose2d(2048, 1024, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (deconv2): ConvTranspose2d(1024, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (deconv3): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (deconv4): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (bn4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (deconv5): ConvTranspose2d(128, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
)

Training on GPU

Check if you can train on GPU. Here, we'll set this as a boolean variable train_on_gpu. Later, you'll be responsible for making sure that

  • Models,
  • Model inputs, and
  • Loss function arguments

Are moved to GPU, where appropriate.

In [14]:
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch

# Check for a GPU
train_on_gpu = torch.cuda.is_available()
if not train_on_gpu:
    print('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Training on GPU!')
Training on GPU!

Discriminator and Generator Losses

Now we need to calculate the losses for both types of adversarial networks.

Discriminator Losses

  • For the discriminator, the total loss is the sum of the losses for real and fake images, d_loss = d_real_loss + d_fake_loss.
  • Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.

Generator Loss

The generator loss will look similar only with flipped labels. The generator's goal is to get the discriminator to think its generated images are real.

Exercise: Complete real and fake loss functions

You may choose to use either cross entropy or a least squares error loss to complete the following real_loss and fake_loss functions.

In [15]:
def real_loss(D_out, smooth=False):
    # compare logits to real labels
    # smooth labels if smooth=True
    #get batch size of output
    batch_size = D_out.size(0)
    if smooth:
       #create row of torch 1's as pos labels
        labels = torch.ones(batch_size) * 0.9
    else:
          #create row of torch 1's as pos labels
        labels = torch.ones(batch_size)
   #define loss method
    labels = labels.cuda()
    criterion = nn.BCEWithLogitsLoss()
    #return loss, squeezing out any empty dims
    return criterion(D_out.squeeze(),labels.squeeze())

def fake_loss(D_out):
    # compare logits to fake labels
    #get batch size
    batch_size = D_out.size(0)
    #make row of zeros as neg label
    labels = torch.zeros(batch_size)
    labels = labels.cuda()
    #define loss method
    criterion = nn.BCEWithLogitsLoss()
    #return loss, squeezing out any empty dims
    return criterion(D_out.squeeze(),labels.squeeze())

Optimizers

Exercise: Define optimizers for your Discriminator (D) and Generator (G)

Define optimizers for your models with appropriate hyperparameters.

In [16]:
import torch.optim as optim
# Create optimizers for the discriminator D and generator G
#use default lr for Adam
d_optimizer = optim.Adam(D.parameters(),lr= 0.0001 ,betas=[0.5,0.999])
g_optimizer = optim.Adam(G.parameters(),lr = 0.0001 , betas=[0.5,0.999])
In [17]:
def view_samples(epoch, samples):
    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
    for ax, img in zip(axes.flatten(), samples[epoch]):
        img = img.detach().cpu().numpy()
        img = np.transpose(img, (1, 2, 0))
        img = ((img + 1)*255 / (2)).astype(np.uint8)
        ax.xaxis.set_visible(False)
        ax.yaxis.set_visible(False)
        im = ax.imshow(img.reshape((32,32,3)))

Training

Training will involve alternating between training the discriminator and the generator. You'll use your functions real_loss and fake_loss to help you calculate the discriminator losses.

  • You should train the discriminator by alternating on real and fake images
  • Then the generator, which tries to trick the discriminator and should have an opposing loss function

Saving Samples

You've been given some code to print out some loss statistics and save some generated "fake" samples.

Exercise: Complete the training function

Keep in mind that, if you've moved your models to GPU, you'll also have to move any model inputs to GPU.

In [18]:
def train(D, G, n_epochs, print_every=50):
    '''Trains adversarial networks for some number of epochs
       param, D: the discriminator network
       param, G: the generator network
       param, n_epochs: number of epochs to train for
       param, print_every: when to print and record the models' losses
       return: D and G losses'''
    try:
        # move models to GPU
        if train_on_gpu:
            D.cuda()
            G.cuda()

        # keep track of loss and generated, "fake" samples
        samples = []
        losses = []

        # Get some fixed data for sampling. These are images that are held
        # constant throughout training, and allow us to inspect the model's performance
        sample_size=16
        fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))
        fixed_z = torch.from_numpy(fixed_z).float()
        # move z to GPU if available
        if train_on_gpu:
            fixed_z = fixed_z.cuda()

        # epoch training loop
        for epoch in range(n_epochs):

            # batch training loop
            for batch_i, (real_images, _) in enumerate(celeba_train_loader):

                batch_size = real_images.size(0)
                real_images = scale(real_images).cuda()

                # ===============================================
                #         YOUR CODE HERE: TRAIN THE NETWORKS
                # ===============================================
                
                # 1. Train the discriminator on real and fake images
                #clear optimizer gradients
                d_optimizer.zero_grad()
                # Generate latent vector
                z = np.random.uniform(-1, 1, size=(batch_size, z_size))
                z = torch.from_numpy(z).float().cuda()
                #generate fake img
                fake_img = G(z)
                #put real image thru discrimator
                real_img_d= D(real_images)
                #Put fake image thru discrimator
                fake_img_d = D(fake_img)
                #generate real loss with smoothing
                real_l = real_loss(real_img_d,smooth=True)
                #generate fake loss
                fake_l = fake_loss(fake_img_d)
                #sum up losses
                d_loss = real_l + fake_l
                #backprop thru network
                d_loss.backward()
                #take optimization step
                d_optimizer.step()

                # 2. Train the generator with an adversarial loss
                g_optimizer.zero_grad()
                
                # Generate latent vector
                z = np.random.uniform(-1, 1, size=(batch_size, z_size))
                z = torch.from_numpy(z).float().cuda()
                #generate fake img
                fake_img = G(z)
                #put fake image thru discrimanator
                fake_img_d = D(fake_img)
                #generate real loss on fake_img
                real_l = real_loss(fake_img_d,smooth=True)
                #sum up losses
                g_loss = real_l
                #backprop thru network
                g_loss.backward()
                #take optimization step
                g_optimizer.step()
                
                
                # ===============================================
                #              END OF YOUR CODE
                # ===============================================

                # Print some loss stats
                if batch_i % print_every == 0:
                    # append discriminator loss and generator loss
                    losses.append((d_loss.item(), g_loss.item()))
                    # print discriminator and generator loss
                    print('Epoch [{:5d}/{:5d}] | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(
                            epoch+1, n_epochs, d_loss.item(), g_loss.item()))


            ## AFTER EACH EPOCH##    
            # this code assumes your generator is named G, feel free to change the name
            # generate and save sample, fake images
            G.eval() # for generating samples
            samples_z = G(fixed_z)
            samples.append(samples_z)
            G.train() # back to training mode
            #view images
            view_samples(epoch,samples)
        # Save training generator samples
        with open('train_samples.pkl', 'wb') as f:
            pkl.dump(samples, f)
        
        # finally return losses
        return losses
    except:
        with open('train_samples.pkl', 'wb') as f:
            pkl.dump(samples, f)
        return losses

Set your number of training epochs and train your GAN!

In [19]:
# set number of epochs 
n_epochs = 15


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# call training function
losses = train(D, G, n_epochs=n_epochs)
08 | g_loss: 6.9360
Epoch [   27/   50] | d_loss: 0.3276 | g_loss: 6.7260
Epoch [   27/   50] | d_loss: 0.3275 | g_loss: 6.7249
Epoch [   28/   50] | d_loss: 0.3307 | g_loss: 6.6917
Epoch [   28/   50] | d_loss: 0.3282 | g_loss: 6.7380
Epoch [   28/   50] | d_loss: 0.3270 | g_loss: 6.8646
Epoch [   28/   50] | d_loss: 0.3278 | g_loss: 6.8100
Epoch [   28/   50] | d_loss: 0.3285 | g_loss: 6.8214
Epoch [   28/   50] | d_loss: 0.3281 | g_loss: 6.8435
Epoch [   28/   50] | d_loss: 0.3283 | g_loss: 6.7573
Epoch [   28/   50] | d_loss: 0.3285 | g_loss: 6.8252
Epoch [   28/   50] | d_loss: 0.3269 | g_loss: 7.1994
Epoch [   28/   50] | d_loss: 0.3267 | g_loss: 6.7816
Epoch [   28/   50] | d_loss: 0.3271 | g_loss: 6.9095
Epoch [   28/   50] | d_loss: 0.3329 | g_loss: 6.8118
Epoch [   28/   50] | d_loss: 0.3292 | g_loss: 6.9006
Epoch [   28/   50] | d_loss: 0.3306 | g_loss: 6.9558
Epoch [   28/   50] | d_loss: 0.3354 | g_loss: 6.9871
Epoch [   28/   50] | d_loss: 0.3266 | g_loss: 7.0694
Epoch [   29/   50] | d_loss: 0.3282 | g_loss: 6.7467
Epoch [   29/   50] | d_loss: 0.3275 | g_loss: 6.8570
Epoch [   29/   50] | d_loss: 0.3270 | g_loss: 6.9154
Epoch [   29/   50] | d_loss: 0.3280 | g_loss: 6.7177
Epoch [   29/   50] | d_loss: 0.3269 | g_loss: 6.6046
Epoch [   29/   50] | d_loss: 0.3267 | g_loss: 6.7627
Epoch [   29/   50] | d_loss: 0.3272 | g_loss: 6.8797
Epoch [   29/   50] | d_loss: 0.3264 | g_loss: 6.9689
Epoch [   29/   50] | d_loss: 0.3310 | g_loss: 6.7967
Epoch [   29/   50] | d_loss: 0.3330 | g_loss: 7.1092
Epoch [   29/   50] | d_loss: 0.3277 | g_loss: 7.0114
Epoch [   29/   50] | d_loss: 0.3285 | g_loss: 6.9560
Epoch [   29/   50] | d_loss: 0.3271 | g_loss: 7.0526
Epoch [   29/   50] | d_loss: 0.3280 | g_loss: 6.9161
Epoch [   29/   50] | d_loss: 0.3261 | g_loss: 6.9440
Epoch [   29/   50] | d_loss: 0.3307 | g_loss: 6.7761
Epoch [   30/   50] | d_loss: 0.3312 | g_loss: 6.8287
Epoch [   30/   50] | d_loss: 0.3275 | g_loss: 6.9219
Epoch [   30/   50] | d_loss: 0.3318 | g_loss: 6.8790
Epoch [   30/   50] | d_loss: 0.3284 | g_loss: 6.8146
Epoch [   30/   50] | d_loss: 0.3303 | g_loss: 6.7072
Epoch [   30/   50] | d_loss: 0.3337 | g_loss: 6.9176
Epoch [   30/   50] | d_loss: 0.3275 | g_loss: 7.4612
Epoch [   30/   50] | d_loss: 0.3289 | g_loss: 6.8657
Epoch [   30/   50] | d_loss: 0.3302 | g_loss: 7.1133
Epoch [   30/   50] | d_loss: 0.3280 | g_loss: 6.9710
Epoch [   30/   50] | d_loss: 0.3281 | g_loss: 6.6765
Epoch [   30/   50] | d_loss: 0.3318 | g_loss: 7.1294
Epoch [   30/   50] | d_loss: 0.3268 | g_loss: 6.8716
Epoch [   30/   50] | d_loss: 0.3271 | g_loss: 6.9070
Epoch [   30/   50] | d_loss: 0.3280 | g_loss: 7.0569
Epoch [   30/   50] | d_loss: 0.3308 | g_loss: 7.2573
Epoch [   31/   50] | d_loss: 0.3303 | g_loss: 7.1032
Epoch [   31/   50] | d_loss: 0.3334 | g_loss: 7.1051
Epoch [   31/   50] | d_loss: 0.3276 | g_loss: 7.3644
Epoch [   31/   50] | d_loss: 0.3270 | g_loss: 7.3440
Epoch [   31/   50] | d_loss: 0.3291 | g_loss: 6.8910
Epoch [   31/   50] | d_loss: 0.3275 | g_loss: 7.2444
Epoch [   31/   50] | d_loss: 0.3268 | g_loss: 7.1494
Epoch [   31/   50] | d_loss: 0.3267 | g_loss: 7.3691
Epoch [   31/   50] | d_loss: 0.3286 | g_loss: 6.9260
Epoch [   31/   50] | d_loss: 0.3282 | g_loss: 6.8922
Epoch [   31/   50] | d_loss: 0.3274 | g_loss: 7.2261
Epoch [   31/   50] | d_loss: 0.3271 | g_loss: 7.0349
Epoch [   31/   50] | d_loss: 0.3278 | g_loss: 7.2673
Epoch [   31/   50] | d_loss: 0.3298 | g_loss: 6.8396
Epoch [   31/   50] | d_loss: 0.3268 | g_loss: 6.8983
Epoch [   31/   50] | d_loss: 0.3288 | g_loss: 7.1496
Epoch [   32/   50] | d_loss: 0.3352 | g_loss: 7.1233
Epoch [   32/   50] | d_loss: 0.3266 | g_loss: 6.9830
Epoch [   32/   50] | d_loss: 0.3263 | g_loss: 7.3946
Epoch [   32/   50] | d_loss: 0.3278 | g_loss: 7.0193
Epoch [   32/   50] | d_loss: 0.3278 | g_loss: 6.7040
Epoch [   32/   50] | d_loss: 0.3271 | g_loss: 6.9962
Epoch [   32/   50] | d_loss: 0.3267 | g_loss: 6.5405
Epoch [   32/   50] | d_loss: 0.3283 | g_loss: 6.9162
Epoch [   32/   50] | d_loss: 0.3286 | g_loss: 7.3080
Epoch [   32/   50] | d_loss: 0.3263 | g_loss: 7.2747
Epoch [   32/   50] | d_loss: 0.3264 | g_loss: 7.3098
Epoch [   32/   50] | d_loss: 0.3270 | g_loss: 7.3431
Epoch [   32/   50] | d_loss: 0.3279 | g_loss: 7.1649
Epoch [   32/   50] | d_loss: 0.3271 | g_loss: 7.1668
Epoch [   32/   50] | d_loss: 0.3262 | g_loss: 7.5403
Epoch [   32/   50] | d_loss: 0.3272 | g_loss: 7.0168
Epoch [   33/   50] | d_loss: 0.3272 | g_loss: 6.9325
Epoch [   33/   50] | d_loss: 0.3273 | g_loss: 7.2034
Epoch [   33/   50] | d_loss: 0.3271 | g_loss: 7.3275
Epoch [   33/   50] | d_loss: 0.3281 | g_loss: 7.0170
Epoch [   33/   50] | d_loss: 0.3268 | g_loss: 6.9334
Epoch [   33/   50] | d_loss: 0.3276 | g_loss: 7.0446
Epoch [   33/   50] | d_loss: 0.3280 | g_loss: 7.4700
Epoch [   33/   50] | d_loss: 0.3287 | g_loss: 7.2275
Epoch [   33/   50] | d_loss: 0.3273 | g_loss: 7.2614
Epoch [   33/   50] | d_loss: 0.3281 | g_loss: 6.5415
Epoch [   33/   50] | d_loss: 0.3276 | g_loss: 6.4814
Epoch [   33/   50] | d_loss: 0.3280 | g_loss: 6.3197
Epoch [   33/   50] | d_loss: 0.3314 | g_loss: 6.7962
Epoch [   33/   50] | d_loss: 0.3269 | g_loss: 6.8227
Epoch [   33/   50] | d_loss: 0.3287 | g_loss: 6.4909
Epoch [   33/   50] | d_loss: 0.3283 | g_loss: 6.9055
Epoch [   34/   50] | d_loss: 0.3287 | g_loss: 6.7255
Epoch [   34/   50] | d_loss: 0.3273 | g_loss: 6.7235
Epoch [   34/   50] | d_loss: 0.3269 | g_loss: 7.1621
Epoch [   34/   50] | d_loss: 0.3284 | g_loss: 6.8765
Epoch [   34/   50] | d_loss: 0.3281 | g_loss: 6.7096
Epoch [   34/   50] | d_loss: 0.3268 | g_loss: 6.7261
Epoch [   34/   50] | d_loss: 0.3278 | g_loss: 6.7756
Epoch [   34/   50] | d_loss: 0.3278 | g_loss: 6.8324
Epoch [   34/   50] | d_loss: 0.3305 | g_loss: 6.8435
Epoch [   34/   50] | d_loss: 0.3264 | g_loss: 7.1204
Epoch [   34/   50] | d_loss: 0.3292 | g_loss: 7.2118
Epoch [   34/   50] | d_loss: 0.3320 | g_loss: 7.1658
Epoch [   34/   50] | d_loss: 0.3268 | g_loss: 8.2512
Epoch [   34/   50] | d_loss: 0.3967 | g_loss: 6.9988
Epoch [   34/   50] | d_loss: 0.3399 | g_loss: 5.8807
Epoch [   34/   50] | d_loss: 0.3283 | g_loss: 6.8844
Epoch [   35/   50] | d_loss: 0.3386 | g_loss: 6.0126
Epoch [   35/   50] | d_loss: 0.3341 | g_loss: 6.9960
Epoch [   35/   50] | d_loss: 0.3316 | g_loss: 6.0464
Epoch [   35/   50] | d_loss: 0.3306 | g_loss: 6.3658
Epoch [   35/   50] | d_loss: 0.3282 | g_loss: 6.0297
Epoch [   35/   50] | d_loss: 0.3305 | g_loss: 6.6289
Epoch [   35/   50] | d_loss: 0.3283 | g_loss: 6.1964
Epoch [   35/   50] | d_loss: 0.3265 | g_loss: 7.7341
Epoch [   35/   50] | d_loss: 0.3289 | g_loss: 7.5471
Epoch [   35/   50] | d_loss: 0.3295 | g_loss: 6.4134
Epoch [   35/   50] | d_loss: 0.3314 | g_loss: 7.3095
Epoch [   35/   50] | d_loss: 0.3286 | g_loss: 7.2151
Epoch [   35/   50] | d_loss: 0.3339 | g_loss: 7.1250
Epoch [   35/   50] | d_loss: 0.3285 | g_loss: 7.8089
Epoch [   35/   50] | d_loss: 0.3265 | g_loss: 7.5128
Epoch [   35/   50] | d_loss: 0.3269 | g_loss: 6.6367
Epoch [   36/   50] | d_loss: 0.3304 | g_loss: 7.0643
Epoch [   36/   50] | d_loss: 0.3275 | g_loss: 7.9753
Epoch [   36/   50] | d_loss: 0.3290 | g_loss: 7.0248
Epoch [   36/   50] | d_loss: 0.3271 | g_loss: 7.3011
Epoch [   36/   50] | d_loss: 0.3292 | g_loss: 7.9477
Epoch [   36/   50] | d_loss: 0.3279 | g_loss: 7.1587
Epoch [   36/   50] | d_loss: 0.3274 | g_loss: 6.8283
Epoch [   36/   50] | d_loss: 0.3274 | g_loss: 6.9848
Epoch [   36/   50] | d_loss: 0.3268 | g_loss: 6.9800
Epoch [   36/   50] | d_loss: 0.3272 | g_loss: 7.4541
Epoch [   36/   50] | d_loss: 0.3266 | g_loss: 7.0388
Epoch [   36/   50] | d_loss: 0.3266 | g_loss: 7.4360
Epoch [   36/   50] | d_loss: 0.3295 | g_loss: 7.1827
Epoch [   36/   50] | d_loss: 0.3291 | g_loss: 7.0967
Epoch [   36/   50] | d_loss: 0.3389 | g_loss: 8.3726
Epoch [   36/   50] | d_loss: 0.3287 | g_loss: 7.5031
Epoch [   37/   50] | d_loss: 0.3314 | g_loss: 6.7812
Epoch [   37/   50] | d_loss: 0.3270 | g_loss: 7.0288
Epoch [   37/   50] | d_loss: 0.3277 | g_loss: 7.9506
Epoch [   37/   50] | d_loss: 0.3298 | g_loss: 8.2282
Epoch [   37/   50] | d_loss: 0.3273 | g_loss: 8.9887
Epoch [   37/   50] | d_loss: 0.3265 | g_loss: 7.9514
Epoch [   37/   50] | d_loss: 0.3273 | g_loss: 7.5165
Epoch [   37/   50] | d_loss: 0.3275 | g_loss: 7.4951
Epoch [   37/   50] | d_loss: 0.3270 | g_loss: 7.3097
Epoch [   37/   50] | d_loss: 0.3288 | g_loss: 7.2246
Epoch [   37/   50] | d_loss: 0.3286 | g_loss: 6.0327
Epoch [   37/   50] | d_loss: 0.3274 | g_loss: 6.9060
Epoch [   37/   50] | d_loss: 0.3308 | g_loss: 8.3933
Epoch [   37/   50] | d_loss: 0.3288 | g_loss: 7.4053
Epoch [   37/   50] | d_loss: 0.3274 | g_loss: 7.1174
Epoch [   37/   50] | d_loss: 0.3339 | g_loss: 7.6723
Epoch [   38/   50] | d_loss: 0.3281 | g_loss: 7.4615
Epoch [   38/   50] | d_loss: 0.3305 | g_loss: 7.4876
Epoch [   38/   50] | d_loss: 0.3265 | g_loss: 7.7496
Epoch [   38/   50] | d_loss: 0.3302 | g_loss: 8.4932
Epoch [   38/   50] | d_loss: 0.3286 | g_loss: 7.1180
Epoch [   38/   50] | d_loss: 0.3267 | g_loss: 7.3927
Epoch [   38/   50] | d_loss: 0.3271 | g_loss: 6.8092
Epoch [   38/   50] | d_loss: 0.3288 | g_loss: 7.1796
Epoch [   38/   50] | d_loss: 0.3281 | g_loss: 7.5568
Epoch [   38/   50] | d_loss: 0.3280 | g_loss: 7.6627
Epoch [   38/   50] | d_loss: 0.3276 | g_loss: 7.8950
Epoch [   38/   50] | d_loss: 0.3270 | g_loss: 8.1540
Epoch [   38/   50] | d_loss: 0.3291 | g_loss: 7.4116
Epoch [   38/   50] | d_loss: 0.3285 | g_loss: 7.2688
Epoch [   38/   50] | d_loss: 0.3300 | g_loss: 7.3689
Epoch [   38/   50] | d_loss: 0.3286 | g_loss: 6.9683
Epoch [   39/   50] | d_loss: 0.3288 | g_loss: 7.2314
Epoch [   39/   50] | d_loss: 0.3265 | g_loss: 7.3259
Epoch [   39/   50] | d_loss: 0.3279 | g_loss: 7.6995
Epoch [   39/   50] | d_loss: 0.3261 | g_loss: 8.7918
Epoch [   39/   50] | d_loss: 0.3280 | g_loss: 7.9527
Epoch [   39/   50] | d_loss: 0.3268 | g_loss: 8.0548
Epoch [   39/   50] | d_loss: 0.3284 | g_loss: 7.6568
Epoch [   39/   50] | d_loss: 0.3266 | g_loss: 7.9079
Epoch [   39/   50] | d_loss: 0.3263 | g_loss: 7.7671
Epoch [   39/   50] | d_loss: 0.3319 | g_loss: 7.1709
Epoch [   39/   50] | d_loss: 0.3286 | g_loss: 8.0461
Epoch [   39/   50] | d_loss: 0.3289 | g_loss: 8.7089
Epoch [   39/   50] | d_loss: 0.3272 | g_loss: 8.3281
Epoch [   39/   50] | d_loss: 0.3264 | g_loss: 7.3462
Epoch [   39/   50] | d_loss: 0.3271 | g_loss: 7.5868
Epoch [   39/   50] | d_loss: 0.3338 | g_loss: 7.4214
Epoch [   40/   50] | d_loss: 0.3292 | g_loss: 7.8269
Epoch [   40/   50] | d_loss: 0.3275 | g_loss: 8.1816
Epoch [   40/   50] | d_loss: 0.3260 | g_loss: 7.4634
Epoch [   40/   50] | d_loss: 0.3266 | g_loss: 8.5159
Epoch [   40/   50] | d_loss: 0.3306 | g_loss: 7.7932
Epoch [   40/   50] | d_loss: 0.3273 | g_loss: 8.5408
Epoch [   40/   50] | d_loss: 0.3275 | g_loss: 8.3225
Epoch [   40/   50] | d_loss: 0.3328 | g_loss: 7.8677
Epoch [   40/   50] | d_loss: 0.3310 | g_loss: 7.3936
Epoch [   40/   50] | d_loss: 0.3298 | g_loss: 7.1695
Epoch [   40/   50] | d_loss: 0.3354 | g_loss: 7.3634
Epoch [   40/   50] | d_loss: 0.3273 | g_loss: 8.0861
Epoch [   40/   50] | d_loss: 0.3299 | g_loss: 7.5609
Epoch [   40/   50] | d_loss: 0.3304 | g_loss: 7.1097
Epoch [   40/   50] | d_loss: 0.3313 | g_loss: 7.5046
Epoch [   40/   50] | d_loss: 0.3280 | g_loss: 7.8335
Epoch [   41/   50] | d_loss: 0.3291 | g_loss: 7.9943
Epoch [   41/   50] | d_loss: 0.3276 | g_loss: 7.0520
Epoch [   41/   50] | d_loss: 0.3272 | g_loss: 8.6294
Epoch [   41/   50] | d_loss: 0.3292 | g_loss: 7.4719
Epoch [   41/   50] | d_loss: 0.3292 | g_loss: 8.9438
Epoch [   41/   50] | d_loss: 0.3281 | g_loss: 7.8662
Epoch [   41/   50] | d_loss: 0.3268 | g_loss: 7.4514
Epoch [   41/   50] | d_loss: 0.3279 | g_loss: 7.3918
Epoch [   41/   50] | d_loss: 0.3278 | g_loss: 6.8910
Epoch [   41/   50] | d_loss: 0.3300 | g_loss: 8.4684
Epoch [   41/   50] | d_loss: 0.3340 | g_loss: 7.4531
Epoch [   41/   50] | d_loss: 0.3268 | g_loss: 8.0926
Epoch [   41/   50] | d_loss: 0.3285 | g_loss: 7.8298
Epoch [   41/   50] | d_loss: 0.3280 | g_loss: 8.2594
Epoch [   41/   50] | d_loss: 0.3262 | g_loss: 8.6102
Epoch [   41/   50] | d_loss: 0.3269 | g_loss: 7.9688
Epoch [   42/   50] | d_loss: 0.3301 | g_loss: 8.5269
Epoch [   42/   50] | d_loss: 0.3268 | g_loss: 7.6958
Epoch [   42/   50] | d_loss: 0.3290 | g_loss: 7.3938
Epoch [   42/   50] | d_loss: 0.3301 | g_loss: 7.1842
Epoch [   42/   50] | d_loss: 0.3342 | g_loss: 7.2753
Epoch [   42/   50] | d_loss: 0.3275 | g_loss: 8.0719
Epoch [   42/   50] | d_loss: 0.3286 | g_loss: 8.0554
Epoch [   42/   50] | d_loss: 0.3272 | g_loss: 7.7504
Epoch [   42/   50] | d_loss: 0.3262 | g_loss: 8.0244
Epoch [   42/   50] | d_loss: 0.3267 | g_loss: 8.1040
Epoch [   42/   50] | d_loss: 0.3284 | g_loss: 8.0679
Epoch [   42/   50] | d_loss: 0.3277 | g_loss: 8.0162
Epoch [   42/   50] | d_loss: 0.3271 | g_loss: 8.2647
Epoch [   42/   50] | d_loss: 0.3266 | g_loss: 8.0047
Epoch [   42/   50] | d_loss: 0.3276 | g_loss: 7.8859
Epoch [   42/   50] | d_loss: 0.3273 | g_loss: 7.8678
Epoch [   43/   50] | d_loss: 0.3267 | g_loss: 8.0741
Epoch [   43/   50] | d_loss: 0.3275 | g_loss: 7.5536
Epoch [   43/   50] | d_loss: 0.3268 | g_loss: 7.5989
Epoch [   43/   50] | d_loss: 0.3266 | g_loss: 8.1208
Epoch [   43/   50] | d_loss: 0.3274 | g_loss: 7.9586
Epoch [   43/   50] | d_loss: 0.3261 | g_loss: 8.0862
Epoch [   43/   50] | d_loss: 0.3278 | g_loss: 7.9996
Epoch [   43/   50] | d_loss: 0.3263 | g_loss: 8.1564
Epoch [   43/   50] | d_loss: 0.3307 | g_loss: 8.2266
Epoch [   43/   50] | d_loss: 0.3265 | g_loss: 8.2037
Epoch [   43/   50] | d_loss: 0.3261 | g_loss: 8.0458
Epoch [   43/   50] | d_loss: 0.3270 | g_loss: 7.8891
Epoch [   43/   50] | d_loss: 0.3264 | g_loss: 7.9542
Epoch [   43/   50] | d_loss: 0.3296 | g_loss: 7.2995
Epoch [   43/   50] | d_loss: 0.3273 | g_loss: 7.5225
Epoch [   43/   50] | d_loss: 0.3300 | g_loss: 6.5937
Epoch [   44/   50] | d_loss: 0.3872 | g_loss: 3.0835
Epoch [   44/   50] | d_loss: 0.3381 | g_loss: 5.5731
Epoch [   44/   50] | d_loss: 0.3394 | g_loss: 5.5365
Epoch [   44/   50] | d_loss: 0.3445 | g_loss: 6.2581
Epoch [   44/   50] | d_loss: 0.3294 | g_loss: 6.1764
Epoch [   44/   50] | d_loss: 0.3353 | g_loss: 5.9953
Epoch [   44/   50] | d_loss: 0.3283 | g_loss: 6.1365
Epoch [   44/   50] | d_loss: 0.3302 | g_loss: 6.0506
Epoch [   44/   50] | d_loss: 0.3287 | g_loss: 6.1008
Epoch [   44/   50] | d_loss: 0.3347 | g_loss: 6.2292
Epoch [   44/   50] | d_loss: 0.3290 | g_loss: 6.1507
Epoch [   44/   50] | d_loss: 0.3280 | g_loss: 6.2418
Epoch [   44/   50] | d_loss: 0.3303 | g_loss: 6.3642
Epoch [   44/   50] | d_loss: 0.3273 | g_loss: 6.3920
Epoch [   44/   50] | d_loss: 0.3277 | g_loss: 6.6707
Epoch [   44/   50] | d_loss: 0.3277 | g_loss: 6.8103
Epoch [   45/   50] | d_loss: 0.3290 | g_loss: 6.7373
Epoch [   45/   50] | d_loss: 0.3288 | g_loss: 6.7329
Epoch [   45/   50] | d_loss: 0.3290 | g_loss: 6.7991
Epoch [   45/   50] | d_loss: 0.3309 | g_loss: 6.2758
Epoch [   45/   50] | d_loss: 2.1690 | g_loss: 1.8048
Epoch [   45/   50] | d_loss: 0.3565 | g_loss: 5.7256
Epoch [   45/   50] | d_loss: 0.4456 | g_loss: 5.3621
Epoch [   45/   50] | d_loss: 0.4863 | g_loss: 3.7457
Epoch [   45/   50] | d_loss: 0.5370 | g_loss: 5.6572
Epoch [   45/   50] | d_loss: 0.4472 | g_loss: 3.5999
Epoch [   45/   50] | d_loss: 0.4877 | g_loss: 3.8420
Epoch [   45/   50] | d_loss: 0.3623 | g_loss: 4.5308
Epoch [   45/   50] | d_loss: 0.4311 | g_loss: 3.4329
Epoch [   45/   50] | d_loss: 0.4295 | g_loss: 3.4814
Epoch [   45/   50] | d_loss: 0.5824 | g_loss: 1.6304
Epoch [   45/   50] | d_loss: 0.5534 | g_loss: 3.8059
Epoch [   46/   50] | d_loss: 0.4035 | g_loss: 3.7025
Epoch [   46/   50] | d_loss: 0.6330 | g_loss: 3.7359
Epoch [   46/   50] | d_loss: 0.4472 | g_loss: 4.0754
Epoch [   46/   50] | d_loss: 0.5699 | g_loss: 3.3789
Epoch [   46/   50] | d_loss: 0.5005 | g_loss: 3.2455
Epoch [   46/   50] | d_loss: 0.7207 | g_loss: 2.9371
Epoch [   46/   50] | d_loss: 0.6567 | g_loss: 2.4114
Epoch [   46/   50] | d_loss: 0.8931 | g_loss: 1.9383
Epoch [   46/   50] | d_loss: 0.8104 | g_loss: 4.3542
Epoch [   46/   50] | d_loss: 0.4780 | g_loss: 2.6192
Epoch [   46/   50] | d_loss: 0.4231 | g_loss: 3.0332
Epoch [   46/   50] | d_loss: 1.2461 | g_loss: 1.0790
Epoch [   46/   50] | d_loss: 0.8709 | g_loss: 1.4649
Epoch [   46/   50] | d_loss: 0.4727 | g_loss: 2.2348
Epoch [   46/   50] | d_loss: 0.6917 | g_loss: 3.6003
Epoch [   46/   50] | d_loss: 0.7647 | g_loss: 3.3791
Epoch [   47/   50] | d_loss: 0.4069 | g_loss: 4.0286
Epoch [   47/   50] | d_loss: 0.4073 | g_loss: 4.4151
Epoch [   47/   50] | d_loss: 0.8486 | g_loss: 1.9191
Epoch [   47/   50] | d_loss: 0.6927 | g_loss: 2.7347
Epoch [   47/   50] | d_loss: 0.6787 | g_loss: 2.8094
Epoch [   47/   50] | d_loss: 0.9819 | g_loss: 2.3136
Epoch [   47/   50] | d_loss: 0.4560 | g_loss: 4.4657
Epoch [   47/   50] | d_loss: 0.7842 | g_loss: 2.0568
Epoch [   47/   50] | d_loss: 0.4726 | g_loss: 3.7604
Epoch [   47/   50] | d_loss: 0.6133 | g_loss: 2.4667
Epoch [   47/   50] | d_loss: 0.7967 | g_loss: 4.0801
Epoch [   47/   50] | d_loss: 0.4172 | g_loss: 2.5548
Epoch [   47/   50] | d_loss: 0.5928 | g_loss: 3.3807
Epoch [   47/   50] | d_loss: 0.7137 | g_loss: 2.1064
Epoch [   47/   50] | d_loss: 1.2330 | g_loss: 1.9528
Epoch [   47/   50] | d_loss: 0.6628 | g_loss: 1.5938
Epoch [   48/   50] | d_loss: 0.5277 | g_loss: 2.6332
Epoch [   48/   50] | d_loss: 0.4598 | g_loss: 4.3157
Epoch [   48/   50] | d_loss: 0.8776 | g_loss: 5.8583
Epoch [   48/   50] | d_loss: 1.0134 | g_loss: 1.8101
Epoch [   48/   50] | d_loss: 0.5944 | g_loss: 2.9626
Epoch [   48/   50] | d_loss: 0.5796 | g_loss: 2.7869
Epoch [   48/   50] | d_loss: 1.1165 | g_loss: 1.4386
Epoch [   48/   50] | d_loss: 1.8866 | g_loss: 4.0050
Epoch [   48/   50] | d_loss: 0.7241 | g_loss: 1.9843
Epoch [   48/   50] | d_loss: 0.7047 | g_loss: 3.7480
Epoch [   48/   50] | d_loss: 0.6414 | g_loss: 2.4150
Epoch [   48/   50] | d_loss: 0.6313 | g_loss: 3.5589
Epoch [   48/   50] | d_loss: 0.7167 | g_loss: 3.4914
Epoch [   48/   50] | d_loss: 0.4841 | g_loss: 3.1409
Epoch [   48/   50] | d_loss: 0.6209 | g_loss: 3.9023
Epoch [   48/   50] | d_loss: 0.5403 | g_loss: 2.4134
Epoch [   49/   50] | d_loss: 0.7336 | g_loss: 3.2792
Epoch [   49/   50] | d_loss: 0.7345 | g_loss: 1.9329
Epoch [   49/   50] | d_loss: 0.6344 | g_loss: 2.5201
Epoch [   49/   50] | d_loss: 0.7136 | g_loss: 2.2521
Epoch [   49/   50] | d_loss: 1.8495 | g_loss: 1.8806
Epoch [   49/   50] | d_loss: 1.0963 | g_loss: 3.1240
Epoch [   49/   50] | d_loss: 0.7160 | g_loss: 1.6277
Epoch [   49/   50] | d_loss: 0.6731 | g_loss: 3.3026
Epoch [   49/   50] | d_loss: 0.5249 | g_loss: 3.6845
Epoch [   49/   50] | d_loss: 0.5032 | g_loss: 4.7567
Epoch [   49/   50] | d_loss: 0.4141 | g_loss: 2.9336
Epoch [   49/   50] | d_loss: 0.6938 | g_loss: 2.1326
Epoch [   49/   50] | d_loss: 0.5391 | g_loss: 2.8570
Epoch [   49/   50] | d_loss: 0.4853 | g_loss: 2.8703
Epoch [   49/   50] | d_loss: 0.4987 | g_loss: 2.6213
Epoch [   49/   50] | d_loss: 0.9410 | g_loss: 4.7149
Epoch [   50/   50] | d_loss: 0.3977 | g_loss: 4.1677
Epoch [   50/   50] | d_loss: 0.6619 | g_loss: 2.0976
Epoch [   50/   50] | d_loss: 0.6824 | g_loss: 4.2163
Epoch [   50/   50] | d_loss: 0.6091 | g_loss: 3.6840
Epoch [   50/   50] | d_loss: 1.9279 | g_loss: 3.5108
Epoch [   50/   50] | d_loss: 0.8889 | g_loss: 3.0801
Epoch [   50/   50] | d_loss: 0.4791 | g_loss: 2.4774
Epoch [   50/   50] | d_loss: 0.6094 | g_loss: 3.2349
Epoch [   50/   50] | d_loss: 0.5114 | g_loss: 2.9293
Epoch [   50/   50] | d_loss: 1.9622 | g_loss: 0.9829
Epoch [   50/   50] | d_loss: 0.6831 | g_loss: 2.5408
Epoch [   50/   50] | d_loss: 0.5096 | g_loss: 3.6064
Epoch [   50/   50] | d_loss: 0.6387 | g_loss: 3.1621
Epoch [   50/   50] | d_loss: 0.6809 | g_loss: 2.7734
Epoch [   50/   50] | d_loss: 0.5150 | g_loss: 2.3012
Epoch [   50/   50] | d_loss: 0.5022 | g_loss: 2.8510

Training loss

Plot the training losses for the generator and discriminator, recorded after each epoch.

In [20]:
fig, ax = plt.subplots()
losses = np.array(losses)
plt.plot(losses.T[0], label='Discriminator', alpha=0.5)
plt.plot(losses.T[1], label='Generator', alpha=0.5)
plt.title("Training Losses")
plt.legend()
Out[20]:
<matplotlib.legend.Legend at 0x26a8718c280>

Generator samples from training

View samples of images from the generator, and answer a question about the strengths and weaknesses of your trained models.

In [15]:
# helper function for viewing a list of passed in sample images
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np

def view_samples(epoch, samples):
    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
    for ax, img in zip(axes.flatten(), samples[epoch]):
        img = img.detach().cpu().numpy()
        img = np.transpose(img, (1, 2, 0))
        img = ((img + 1)*255 / (2)).astype(np.uint8)
        ax.xaxis.set_visible(False)
        ax.yaxis.set_visible(False)
        im = ax.imshow(img.reshape((32,32,3)))
In [16]:
# Load samples from generator, taken while training
with open('train_samples.pkl', 'rb') as f:
    samples = pkl.load(f)
In [19]:
_ = view_samples(-1, samples)

Question: What do you notice about your generated samples and how might you improve this model?

When you answer this question, consider the following factors:

  • The dataset is biased; it is made of "celebrity" faces that are mostly white
  • Model size; larger models have the opportunity to learn more features in a data feature space
  • Optimization strategy; optimizers and number of epochs affect your final result -My generated samples are all white, which caused at a data in the bias , making most of my generated set white aswell. -My model was large, but perhaps not large enough to capture all the nuances of a human face. -Perhaps a higher learning rate would have resulted in better generated faces.

Answer: (Write your answer in this cell)

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "problem_unittests.py" files in your submission.